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OBJECT DETECTION 



Field of the Invention 
The present invention relates to object detection. More specifically, the present 
invention relates to the detection of objects in a user-defined area having a patterned 
background. 

Background of the Invention 
Motion detection and object detection systems are known in the art. Frequently, 
such systems are used to monitor a user-defined area to detect whenever an object 
enters the area. The equipment needed to implement an object detection system is well 
known in the art. Such systems comprise an imaging device (typically a video camera) 
capable of capturing an image of the monitored area and a device for digitizing the 
captured images. The digitized images are then analyzed in an attempt to detect 
whether an object has entered the monitored area. There exist many different known 
methods and algorithms for analyzing digitized images for determining when an object 
has entered a monitored ar6a. Two of the most common methods utilized are generally 
referred to as motion segmentation methods or change detection methods. 

A change detection method of object detection in an image is accomplished by 
examining the difference between a current image and a reference image that contains 
only the static background of the monitored area or scene. A reference image can be 
thought of as a representation of the monitored area as it would appear if no transitory 
objects were in view. Change detection algorithms take two digitized images as input 
and return the locations in the field of view where differences between the images are 
identified. These differences may be caused by the motion of an object in the field of 
view, the addition or removal of an object from the scene, changes in illumination, or 
noise from the digitization process. The objective of change detection algorithms is to 
locate only the changes that are due to structural changes in the scene, that is, an object 
moving or the introduction or removal of an object in the scene. 
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However, many object detection systems are utilized in environments that cause 
shadows to be cast on the monitored area or zone or in environments that experience 
significant changes in ambient lighting conditions. Such environments are often found 
in industrial settings. Shadows may be caused by direct and/or diffuse illumination 
5 from the environment outside the monitored area. Objects moving near the monitored 
area may have their shadows cast into the monitored area. Additionally, shadows may 
change as the lighting conditions in the environment change. Changes in the lighting 
conditions may be due to changes in the amount of illumination or movement of the 
light source. 

Object detection systems need to distinguish actual objects in a monitored area 
from shadows that are being cast into the area and changes in the ambient lighting 
conditions. Existing techniques have attempted to distinguish actual objects in a 
monitored area from shadows that are being cast into the area and changes in the 
ambient lighting conditions, but these existing techniques have enjoyed limited success. 
Therefore, a need continues to exist in the art for object detection systems and methods 
that are superior to existing systems and methods for distinguishing objects from 
shadows and changing lighting conditions. 

Summary of the Invention 
The present invention provides for object detection that is superior to the prior 
art at distinguishing between objects that have moved into a monitored area and 
shadows that have been cast into the monitored area. A patterned background is 
utilized in the present invention. According to the present invention, portions of the 
patterned background are analyzed to determine whether an object exists in any of the 
portions. For purposes of the present specification, each portion of the patterned 
background is referred to as a mask window. Preferably, the size of the mask window 
is designed so that it is no larger than the approximate size of the smallest object for 
which detection is desired. Mask windows are overlapped in a manner so as to cover 
the area for which object detection is desired. The patterned background is designed so 
that each mask window contains both light areas and dark areas. In a preferred 
embodiment, the patterned background is designed so that in each mask window the 
amount of light area and amount of dark area is approximately equal. 
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The present invention takes advantages of certain phenomena that occur when, 
in accordance with the present invention, a live image is compared to a reference image 
exhibiting a patterned background. First, a difference image produced by subtracting 
the reference image from a live image containing an object will contain a complement 
or inverse image of the object. Second, live images containing shadows instead of 
objects tend to not produce any complements or inverse images in a difference image. 

Because the overall background pattern and the position of each mask window 
are known and do not change during object detection, the background pattern within 
each mask window is known and is constant. Thus, certain calculations corresponding 
to each reference image can be made once during initialization of the object detection 
system and then used as constants during analysis of a live image. This use of constants 
calculated at initialization allows for faster image analysis at run time, which in turn 
allows image capture devices with faster frame rates to be used. 

According to the present invention, an object is detected when the difference 
between the expected value for the brightness levels in the portion of the live image 
corresponding to the portion of the reference image containing light pixels and the 
expected value for the brightness levels in the portion of the live image corresponding 
to the portion of the reference image containing dark pixels is less than some threshold 
T. Stated another way, object detection is indicated whenever the following 
20 relationship holds: 

(E[Live L (x,y)J - E[Live D (x,y)]) < T, 
wherein E[Live L (x,y)] is the expected value for the brightness levels in the portion of 
the live image corresponding to the portion of the reference image containing light 
pixels, wherein E[Live D (x,y)] is the expected value for the brightness levels in the 
portion of the live image corresponding to the portion of the reference image containing 
dark pixels, wherein 0 < T < Yref , wherein y ref = (E[Ref L (x,y) -E[Ref D (x,y)]), wherein 
E[Ref L (x,y) is the expected value of the brightness levels of the light pixels in the 
reference image and E[Ref D (x,y)] is the expected value of the brightness levels of the 
dark pixels in the reference image. In a preferred embodiment of the present invention, 
30 T is equal to about y re f/2. 
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As explained above, Yref can be calculated once and T can be chosen once at the 
time when a system embodying the present invention is setup or initialized and then 
both y ref and T can be used as constants throughout the operation of the system. 
Alternatively, the reference image could be updated periodically and new values for y ref 
and T calculated. Updating the reference image periodically would allow for 
embodiments of the present invention to account for changes in the environment, such 
as a slight deterioration of the background pattern over time. 

The present invention can be embodied in a monitoring system able to detect the 
intrusion of people (or other objects) into a user-defined safety zone. The present 
invention provides superior methods for distinguishing between objects actually 
entering the user-defined area and shadows caused to be cast onto the user-defined area 
by changes in lighting or objects moving near the user-defined area. 

Description of the Drawings 

Figure 1 depicts examples of patterns useful in the present invention. 

Figure 2 depicts an embodiment of the present invention. 

Figure 3 depicts a reference image, a live image containing an object (that is, a 
hand), and a reference image produced in accordance with the present invention. 

Figure 4 shows a flowchart embodying a method according to the present 
invention. 

Detailed Description of the Invention 
Unless indicated otherwise, the term "image" used in the current specification 
will refer to a digitized image having an array or vector of numbers representing 
brightness (that is, luminance intensity) levels. Each brightness level ranges from 0 (the 
darkest) to I max (the brightest). In embodiments of the present invention, the actual 
numerical intensity levels may range from 0 to 1 .0 or from 0 (00000000 in binary 
notation) to 255 (1 1 1 1 1 1 1 1 in binary notation), for example, depending on the specific 
equipment utilized. Equipment for digitizing images and converting to arrays of 
brightness levels is well known in the art. The numerical brightness levels are 
frequently referred to as pixels. 
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According to the present invention, each captured image of the monitored area is 
divided up into portions or segments. These portions or segments are referred to as 
mask windows. The size of each mask window is chosen so that it is no bigger than the 
approximate size of the smallest object for which detection is desired. This is referred 
to as the target size. The present invention can detect objects smaller than the mask 
window, but the smaller an object is relative to the mask window, the lower the 
probability will be that the object will be detected. Thus, the size of objects to be 
detected are preferably at least as big as the area represented by the mask window. 

Positions of mask windows are chosen so that the entire area to be monitored is 
covered by overlapping mask windows. Preferably, each mask window overlaps at 
least one other mask window. More preferably, each mask window overlaps 
approximately one half of at least one other mask window. 

For each captured image of the monitored area, the image corresponding to each 
mask window is analyzed for object detection. Generally, object detection according to 
the present invention can be implemented on standard hardware known to one of 
ordinary skill in the art to operate sufficiently fast to complete analysis of all window 
masks for each capture image before the next image of the monitored area is captured. 

The present invention provides for improved object detection in monitored areas 
having a patterned background. The patterned background is designed so that each 
mask window contains both light areas and dark areas. In a preferred embodiment, the 
patterned background is designed so that in each mask window the amount of light area 
and amount of dark area is approximately equal. Figure 1 depicts examples of patterns 
useful in the present invention. The light and dark areas of a pattern are designed so 
that the brightness value or intensity level of the image pixels corresponding to the light 
areas is substantially greater than the brightness value or intensity level of the image 
pixels corresponding to the dark areas. The image pixels corresponding to the dark 
areas will all have values much less than I max , and preferably, they will all have values 
close to zero. Similarly, the image pixels corresponding to the light areas will all have 
values much greater than zero, and preferably, they will all have values close to I max . 

One embodiment of the present invention utilizes a patterned area surrounding 
an area into which objection detection is desired. An example of such an embodiment 



is depicted in Figure 2 along with three overlapping mask windows 11. In actual 
operation, the number of mask windows 11 would be sufficient to allow the entire 
patterned background to be monitored for objects. Each mask window 11 is analyzed 
for the presence of objects. It is assumed that no object can enter the surrounded area 
without passing through the patterned area. Thus, if no object is detected in any of the 
mask windows 11, then no object will have been detected in the patterned area and it 
can be assumed that no object has entered the surrounded area. 

Figure 2 also shows an image capturing device 12, an image analysis device 13, 
and two different types of warning devices: an alarm 14 and a monitor 15. Image 
capturing devices 12 useful in the present invention are known in the art. Image capture 
device 12 may be a camera or other device capable of capturing an image. Various 
attachments can be used with image capture device 12 to enhance or modify the manner 
in which an image is captured. For example, a telescopic lens attached to image capture 
device 12 enables the monitoring of an area at a significant distance from the image 
capture device 12. Generally, images captured by the image capturing device 12 are 
analyzed, and stored if necessary, in the image analysis device 13. Image analysis 
devices 13 useful in the present invention can be any devices capable of carrying out the 
calculations required by the present invention. For example, the image analysis device 
13 can be a computing device, such as a personal computer, programmed to carry out 
the needed calculations. In one embodiment, the image capture device 12 and the 
image analysis device 13 reside in the same hardware. For example, the image capture 
device 12 can be a camera having an on-board or embedded microprocessor used as an 
image analysis device 13. If an object is detected an appropriate signal or indication is 
made. The signal can take the form of sounding and alarm 14 or displaying information 
on a monitor 15. 

The embodiment depicted in Figure 2 can be effectively utilized in an industrial 
setting, for example, where the area of interest is a portion of a floor, perhaps containing 
potentially hazardous equipment. A pattern according to the present invention can be 
painted on the floor surrounding the area of interest. An imaging device is then placed 
in a position (on the ceiling, for example) such that the patterned area can be monitored 
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for objects. If an object is detected in the patterned area, an alarm can be sounded or the 
potentially hazardous equipment shut down. 

The present invention takes advantage of certain characteristics or phenomena 
about the difference between a current or live image and a reference image. A 
difference image can be calculated by subtracting the brightness level of the reference 
image from the brightness level of the live image at each pixel. The calculation of a 
difference image can be represented by the following equation: 

Diff(x,y) = Live(x,y) - Ref(x,y) 
where Live(x,y) represents the brightness level at position (x,y) in a live image, 
Ref(x,y) represents the brightness level at the same position (x,y) in the reference 
image, and Diff(x,y) represents the difference between these two brightness levels. 

According to the present invention, the following phenomena is utilized. 
Whenever an object is located in the live image at a position (x,y), Diff(x,y) will be the 
complement of Ref(x,y). That is, if Ref(x,y) is a light pixel (that is, has a high 
brightness level) then Diff(x,y) will have a lower brightness level than Ref(x,y) and if 
Ref(x,y) is a dark pixel then Diff(x,y) will have a higher brightness level than Ref(x,y). 
Accordingly, Diff(x,y) will be the complement or inverse of Ref(x,y) when an object is 
present in the live image. This phenomena is shown in Figure 3 and can be further 
explained in the following manner: 

Let I L = value of Ref(x,y) when Ref(x,y) is a light pixel, 
Let I D = value of Ref(x,y) when Ref(x,y) is a dark pixel, and 
Let / = the value of Live(x,y). Then, 

Diff(x,y) = (/ - I L ) if Ref(x,y) = I L 
Diff(x,y) = (/ - I D ) if Ref(x,y) = I D 
25 Since I D < I L , the following inequality holds 

(/-I D )>(Z-I L ),V/ 

According to the present invention, the following phenomena is also utilized. 
Let Diff L (x,y) be that portion of the difference image corresponding to the position 
where the pixels in Ref(x,y) are light and let Diff D (x,y) be that portion of the difference 
30 image corresponding to the position where the pixels in Ref(x,y) are dark. When an 
object has entered the mask window the expected value (typically calculated as the 
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average value) of the values in Diff L should be less than the expected value of the 
values in Diff D . This is expressed as follows: 

3 an object E[Diff L (x )y )] < E[Diff D (x,y)] , 
where E[Diff L (x,y)] is the expected value of the pixels in Diff L (x,y) and E[Diff D (x,y)] is 
the expected value of the pixels in Diff D (x,y). Equivalent^, this can be expressed as 
follows: 

E[Diff L (x,y)] > E[Diff D (x,y)] 3 an object 
This equation can be rearranged to produce the following: 

(E[Live L (x,y)] - E[Live D (x,y)]) > (E[Ref L (x,y) -E[Ref D (x,y)]) -» 3 an object 
where Live L (x,y) is that portion of the live image corresponding to the position where 
the pixels in Ref(x,y) are light and let Live D (x,y) be that portion of the live image 
corresponding to the position where the pixels in Ref(x,y) are dark. Since Ref L (x,y) is 
known and is constant, the expected value of Ref L (x,y), E[Ref L (x,y)], is only required to 
be calculated once. This calculation can be done during the configuration or 
15 initialization of the system prior to operating the system. Similarly, E[Ref D (x,y)] can be 
calculated once and then used whenever needed. Let the constant (E[Ref L (x,y) - 
E[Ref D (x,y)]) be represented as y K{ . The above equation then becomes: 
(E[Live L (x,y)] - E[Live D (x,y)]) > y ref * 3 an object 
The present invention also utilizes another phenomenon. If a shadow falls on a 
20 pattern of the present invention it will darken all pixels to some degree, but will not 
cause lighter pixels to have intensity values lower than the darker pixels. That it, the 
expected value of the light pixels in a live image, E[Live L (x,y)], should be greater than 
the expected value of the dark pixels in a live image, E[Live D (x,y)J, even when there is 
a shadow in the mask window. This observation can be expressed by the following 
25 equation. 

(E[Live L (x,y)] - E[Live D (x,y)]) < 0 (3 an object) 

The two equations above cover all cases except those covered by the following 
equation. 

0 < (E[Live L (x,y)J - E[Live D (x,y)]) < Yref 
» In an ideal world, the above condition would be considered an object intrusion since the 
difference in expected values found in the live image is less than the difference in 
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expected values found in the reference image, namely Yref . However, this analysis of 
the above condition does not address any noise effects. Taking noise effects into 
consideration in conjunction with the above three equations, an object is detected when 
the deference in expected values is less than some threshold T, where the value of T is 
5 greater than 0 and less than y ref . 

Thus, according to the present invention, an object is detected when 
(E[Live L (x,y)] - E[Live D (x,y)]) < T, 
where 0 < T < Yref . In a preferred embodiment of the present invention, T is equal to 
about Yref /2. Values of T may be optimized for specific applications without undue 
10 experimentation. 

Once an object is detected, the present invention can indicate or signal that an 
object has been detected in any number of ways known in the art. For example a 
system according to the present invention could send a signal to an alarm system. 
Figure 4 shows a flowchart embodying a method according to the present 
5 invention. It should be understood that Figure 4 is intended to be illustrative not 

restrictive. Many other embodiments of the present invention will be apparent to those 
of skill ,n the art. For example, the particular sequence of steps depicted in Figure 4 
does not imply any temporal ordering of their execution in the broadest embodiments of 
the present invention. 
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